Multimodal Corpus for Psychotherapeutic Situations

This paper presents a design principle for construction of an in-house multimodal corpus for computationally analysing and better understanding conversations during psychotherapy. We discuss some sharable information about research community data collection procedures such as recording devices, consent forms, and privacy considerations. We also explain multimodal coding schema and metadata that are needed in the domain. The corpus has three distinguishing properties: 1) it was constructed only for our research and not for public use; 2) the conversation and recording environment was in actual social situations and not controlled; 3) a multimodal coding schema that focuses on the co-construction nature of the conversation was used. Although the conversation contents are not sharable, the data collection procedure and the schema design for the psychotherapy corpus serve as an example of an initiative to construct a multimodal corpus.